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We investigate the suitability of statistical model checking techniques for analysing quantitative prop¬ 
erties of software product line models with probabilistic aspects. For this purpose, we enrich the 
feature-oriented language FLan with action rates, which specify the likelihood of exhibiting par¬ 
ticular behaviour or of installing features at a specific moment or in a specific order. The enriched 
language (called PFLan) allows us to specify models of software product lines with probabilis¬ 
tic configurations and behaviour, e.g. by considering a PFLan semantics based on discrete-time 
Markov chains. The Maude implementation of PFLan is combined with the distributed statistical 
model checker MultiVeStA to perform quantitative analyses of a simple product line case study. The 
presented analyses include the likelihood of certain behaviour of interest (e.g. product malfunction¬ 
ing) and the expected average cost of products. 


1 Introduction 


The modelling and analysis by means of process calculi and formal verification techniques like model 
checking of the variety of configurations and behaviour that is common to a software product line (SPL) is 
gaining momentum |[5]- (9p6l - [r8l|23p3p8p9| . Compared to the complexity of verifying the behaviour of 
a single product or a single system, the variability inherent to SPL adds another dimension as the number 
of possible products of an SPL may be exponential in the number of features pd| . In ||^, we introduced 
the feature-oriented language FLAN as a high-level modelling language for SPLs. A rich set of process- 
algebraic operators allows one to specify in a procedural, operational way both the configuration and the 
behaviour of products, while a constraint store allows one to specify in a declarative way all common 
structural constraints known from feature models and additional action constraints typical of feature- 
oriented software development. On the one hand, the execution of a process is constrained by the store 
(e.g. to avoid introducing inconsistencies), while on the other hand a process can query the store (e.g. to 
resolve configuration options) or update the store (e.g. to add new features, also at run time or by means 
of a staged configuration process). An implementation of FLAN in the executable modelling language 


Maude 1191 allows one to exploit Maude’s rich toolkit for a variety of formal analyses of FLan models, 
ranging from consistency checking (by means of SAT solving) to model checking. 

In this paper, we introduce a probabilistic extension of FLan : PFLAN allows to equip actions with 
rates to specify probabilistic SPL models (e.g. to model uncertainty, failure rates, randomisation). This 
paves the way for quantitative analyses (e.g. to measure quality of service, reliability, performance). Here 
we present a proof-of-concept use of an implementation of PFLan in Maude in combination with the 
distributed statistical model checker MultiVeStA | |36| to estimate the likelihood of specific behaviour. 
Formally, our approach is to perform a sufficient number of probabilistic simulations of a PFLAN model 
to obtain statistical evidence (with a desired level of statistical confidence) of quantitative properties 
under scrutiny. The properties are formulated in MultiVeStA’s property specification language Multi- 
QuaTEx, which allows to express and evaluate more than one property over the same simulated path 
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(behaviour) | |M| |. The advantage over exhaustive (probabilistic) model checking is that there is no need 
to generate entire state spaces. We argue that this outweighs the main disadvantage of having to give 
up on obtaining exact results (100% confidence) with exact analysis techniques like probabilistic model 
checking, in particular when examining an SPL, given their possibly exponential number of products. 

We refer to Q for (probabilistic) model checking and to |^29| for statistical model checking. An 
overview of related work on applying formal analysis techniques in SPLE can be found in 1381, while Q 
contains an extensive discussion of related work on model-checking SPL behaviour. As far as we know, 
there are only a few, quite different, approaches on probabilistic model checking of an SPL p2l[2^|40| , 
whereas we present here the first application of statistical model checking in SPL engineering (SPLE). 

The paper outline is as follows. Section|^contains a toy example of a product line of coffee machines, 
adapted from (5]^. Section|^presents PEL AN, followed by a PEL AN model of the example in Section|^ 
MultiVeStA is introduced in Section]^ followed by experimental quantitative analyses of the example in 
Section]^ Section [^summarises the contributions of this paper and discusses future work. 


2 An Example Product Line of Coffee Machines 


Our toy example is a (simplistic) product line of coffee machines with the following list of requirements: 

1. Initially, a coin must be inserted: either a euro, exclusively for products for the European market, 
or a dollar, exclusively for Canadian products; 

2. An optional cancel button allows the user to cancel coin insertion, after which the coin is returned; 

3. A machine that contains a coin must offer a choice to add sugar, followed by a choice of beverages; 

4. The choice of drinks (coffee, tea, cappuccino) varies, but all products must offer at least one drink, 
tea may be offered only by European products, and products offering cappuccino must offer coffee; 

5. An optional ringtone may be rung after beverage delivery. It must be rung after serving cappuccino; 

6. After the drink is taken, the machine returns idle. 


These requirements for products combine structural constraints defining valid feature configurations (e.g. 
“every product must offer at least one beverage ”) with temporal constraints defining valid producf be¬ 
haviour in terms of valid action sequences (e.g. “a ringtone must be rung after serving a cappuccino”). 

The de facto standard variability model in SPLE is a feature model p7||M |. It provides a compact rep¬ 
resentation of all valid products of a product line in terms of their features (behaviour is not captured). An 
(attributed) feature model of our example is depicted in Eig. [T] It has a root (feature) m and a set of non¬ 
trivial features, partitioned into the sets {b^o} of compound features and Features = {s,r,x,p,c,t,d,e} 
of primitive features^ The only purpose of the former is to group the (primitive) features in the tree, 
whereas the latter define user observable configuralion paramefers @0- We identify a product from 
the product line with a non-empty subset of Features. Deciding whether a product satisfies a feature 
model can be reduced to Boolean satisfiability (SAT), and efficiently be computed with SAT solvers [*4|. 

By equipping features with (non-functional) attributes (e.g. cost{Tea) = 3) we obtain an attributed 
feature modcZj^The cost function cost : Features —)■ N, associated to the attribute cost, straightforwardly 
extends to products: costiproduct) = { cost(feature) \ feature G product}. Thus, intuitively, cost can be 

seen as a labelling function assigning a non-negative integer to each product defined by a feature model. 


Tn case no confusion can arise, we often simply speak of features when we actually refer to the primitive features. 

^ Additional quantitative constraints on (combinations of) features may be defined (e.g. cost{Sugar) -f cost{Ringtone) < 
cost{Com)) but we prefer to neglect them in this paper, as such constraints require the use of SMT solvers like Microsoft’s 
Z3 |30[, currently under integration in our framework. 
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Figure 1: Attributed feature model of Coffee Machine (with shorthand names) 


3 PFLan: Syntax and Semantics 


The feature-oriented language PFLan is a probabilistic extension of FLAN Q, a process algebra that 
neatly separates declarative (pre-)configuration from procedural run-time aspects. PFLAN is inspired 
by the concurrent constraint programming paradigm of p2|, its adoption in process calculi p4| and its 


stochastic extension |12|. A constraint store allows one to specify all common constraints known from 
feature models in a declarative way, while a rich set of process-algebraic operators allow to specify the 
configuration and behaviour of product lines in a procedural way. The semantics smoothly unifies static 
(pre-configurafion) and dynamic (run-fime) feafure selection. 

The core notions of PFLAN are features, constraints, processes (wifh action rates) and fragments, all 
visible in ifs synfax in Fig.|^ More precisely, / and g range over fealures while fhe synfacfic cafegories F, 
S and P correspond fo fragmenls, a consfrainf store and processes (wifh acfions from A), respecfively. 

The universe of (primifive) fealures is denoted by The fealures of our example are fhe accepted 
coin slofs (i.e. euro and dollar), fhe offered drinks (i.e. coffee, tea and cappuccino) and fhe additional 
capabilities sugar, cancel and ringtone (to add sugar, cancel coin insertion and ring a tone, respecfively). 

The declarafive part of PFLAN is represented by a sfore of conslrainls on fealures exlracled from 
fhe producl line requiremenls plus some addilional informalion (e.g. aboul fhe confexl wherein fhe prod- 
ucl will operate). Two imporfanf notions of a consfrainf sfore S are fhe consistency of S, denoted by 
consistentS) (which in our case amounls to logical salisfiabilily of all conslrainls consliluling S) and fhe 
entailment 5 h c of consfrainf c in 5 (which in our case amounls lo logical enfailmenl). A consfrainf sfore 
confains any ferm generated by S according to fhe synfax of PFLAN. The mosl basic consfrainf sfores 
are T (no conslrainls al all), _L (inconsisfenf conslrainls) and ordinary Boolean proposilions (generated 
by K). Conslrainls can be combined by juxfaposifion (ifs semantics amounls fo logical conjunction). 


We assume fhal conslrainls on fealures are expressed using Boolean propositions (cf. |34|). More¬ 
over, we assume fhal fhe universe of proposilions confains a Boolean predicate has{f) fhal can be 
used to denole fhe presence of a feafure / in a producl. Boolean proposilions can also be used fo rep- 
resenl addilional informalion such as conlexlual facls. In our example we use fhe Boolean proposilions 
in{Europe) and in{Canada) lo slale fhe facl fhal fhe coffee machine being configured is meanl fo be 
used in Europe or in Canada, respectively. Finally, Boolean proposilions can slale relations belween 
conlexlual information and fealures, like has{euro) —)• in{Europe) (i.e. a coffee machine has a coin slol 
for euro’s only if il is intended for Ihe European markel). 
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F : 

:= [S\P] 

S,T : 

:= ^ 1 />g 1 f®g 1 SP 1 T 1 A 

m ■ 

:= 0 1 A 1 (A,r).P | P + Q | P;G | P|| 2 

A : 

:= a install(/) ask(P') 

K : 

:= p\^K \ KV K 


Figure 2: Syntax of PFLan, where r G a G p ^ 3^ and f,g € ^ 


Two eommon eross-tree constraints are instead handled as first-class citizens in PFLan. A constraint 
f t>g expresses that feature / requires the presence of feature g, whereas a constraint / <8) g expresses 
that features / and g mutually exclude each other’s presence (i.e. they are incompatible). Also these 
constraints could of course be encoded as Boolean propositions (e.g. f^g and /og can equivalently be 
expressed as has{f) -H- ->has{g) and has{f) —)• has{g), respectively). We in fact use such logical encod¬ 
ings to reduce consistency checking and entailment to logical satisfiability (and hence exploit Maude’s 
SAT solver). However, we prefer to keep this first-class treatment as syntactic sugar. In our example, we 
extract dollar®euro to formalise that euro and dollar are mutually exclusive features (requirement 1) 
and cappuccino t> coffee to formalise that cappuccino requires coffee (requirement 3). 

We also consider a class of action constraints, reminiscent of featured transition systems (FTS) 1181. 
In an FTS, transitions are labelled with actions and with Boolean constraints over the set of features. 
We associate arbitrary constraints to actions rather than to transitions (and we moreover add a rate to 
the actions, discussed below). In a coffee machine offering coffee, e.g., we will use coffee for the (user) 
action of choosing coffee and do{coffee) as a proposition stating the execution of that action. The relation 
between the action coffee and the presence of the corresponding feature coffee can be formalised as 
do{coffee) —)• has{coffee), i.e. the choice for coffee requires coffee being offered by the coffee machine. 
In general, we assume that each action a may have a constraint do{a) —)■ p, where p G is a proposition. 
Such constraints act as a kind of guards to allow or forbid the execution of actions (cf. the discussion of 
the rule ACT below). Note that these action constraints could also be more complex, e.g. we could define 
an action cafe-au-lait together with the action constraint do{cafe-au-lait) —)> {has{coffee) f\has{milk)). 


The procedural part of PFLan is represented by processes which can be of the following type: 

0 the empty process that does nothing; 

X a process identifier0 

(A, r) .P a process that can perform action A with rate r and then behaves as P; 

P + Q a process that can non-deterministically choose to behave as either P or Q\ 

P,Q a process that must progress first as P and then as 2; 

P \\Q a process formed by the parallel composition of P and Q, which evolve independently. 

We distinguish ordinary actions from a universe s/ and two special actions install(/) (which will be used 
to denote the dynamic installation of a feature /) and ask(P') (which can used to query the store for the 
validity of a Boolean proposition from K). As we will see shortly, each action type is treated differently 
in the operational semantics. Note, moreover, that each action has an associated rate (sometimes called 
weight), which is used to determine the probability that this action is executed. As usual, the probability 
to execute an action in a certain state depends on the rates of all other actions enabled in the same state. 

^ We assume there is a set of process definitions of the form X = P and recursively defined processes to be finitely branching. 
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(Inst) 


consistent{S has{f)) 


[S I (install(/),r).P] [S has{f) \ P] 
SPK 


(Ask) 

(Act) 


[5| (ask(i5:),r).P] -^[S\P] 
S = {do{a)^K) ShK 
[S I {a,r).P] ^ [5 I P] 


(Or) 

(Seq) 

(Par) 


[5 I P] ^ [S' I P'] 
[5|P + <2] ^ [S'\P'] 
[5 I P] ^ [S' I P'] 
[5|P;e] ^ [S'[P'-,Q] 
[5 I P] ^ [S' I P'] 


[5 I P II 2] ^ [S' I P' II Q] 


Figure 3: Reduction semantics of PFLan 


P+{Q + R) = 

{P + Q)+R 

P + ib = P 

P + Q = 

Q + P 

^11(211/?) = 

(P II Q) II R 

P||0 = P 

^lie = 

Q\\P 

P-,{Q-,R) = 

{P\Q)\R 

P\tb = P = 0;P 

p = 

P[Q/x] if X = Q 


Figure 4: Structural congruence in PFLan 


We will illustrate this in our example in SectionWe consider the actions euro, dollar (respective coin 
insertion), cancel (cancellation of coin insertion), sugar (sugar selection), ringtone (ringtone emission), 
cojfee, tea and cappuccino (beverage selection) in our example. Their associated rates are discussed 
below. For simplicity we consider only constant rates, but our framework can be easily extended to allow 
store-dependent rates (e.g. to be able to reflect a higher probability to order cappuccino in Europe). 

Finally, a fragment P is a term [S | P], composed by a constraint store S and a process P. These two 
components may influence each other according to the concurrent constraint programming paradigm p2| : 
a process may update its store which, in turn, may condition the execution of the process’ actions. 


The operational semantics is formalised in terms of the state transition relation —;■ C N'^xR+xF 
in Fig.|^ where F denotes the set of all terms generated by F in the grammar of Fig.|^ Note that multisets 
of transitions are needed to deal with the possibility of having multiple instances of a transition F ^ G. 
Technically, such a reduction relation is defined in structural operational semantics (SOS) style (i.e. by 
induction on the structure of the terms denoting a fragment) modulo the structural congruence relation 
= C F X F defined in Fig. The reduction relation implicitly defines a labeled transition system LTS, 
whose labels are rates. It is straightforward to obtain a discrete time Markov chain (DTMC) from such 
LTSs by normalising the rates into [0..1] such that in each state, the sum of the rates of its outgoing 
transitions equals one. As usual, in the resulting DTMC the label of a transition corresponds to the 
probability that such a transition is executed starting from its source state. Recall that we advocate the 
use of statistical model checking because in general the DTMC is too large to generate. 

The rules iNST and Act of the semantics are very similar, both allowing a process to execute an 
action if certain constraints are satisfied. Rule iNST forbids inconsistencies caused by the introduction 
of new features. It can be seen as a particular instance of the rule for the tell operation of concurrent 
constraint programming |32| instantiated as ].e\\{has{f)). Rule Act forbids inconsistencies with respect 
to action constraints. A typical action constraint is do{a) —)> has{f), i.e. action a is subject to the presence 
of feature /. Rule Ask formalises the semantics of the ask(-) operation from concurrent constraint 
programming | [32| . It allows a process to be blocked until a proposition can be derived from the store. 
Rules Par, Seq and Or formalise interleaving parallel composition, sequential composition and non- 
deterministic choice, respectively. Note that the non-determinism introduced by choices and parallel 
composition is probabilistically resolved in the aforementioned DTMC semantics. 
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Summarising, we note a variety of ways in which a feature / can be included in a configuration. First, 
an explicit and declarative way is to include the proposition has{f) in the initial store; this is the way 
to include core features. Second, an implicit and declarative way is to derive / from other constraints 
(e.g. if a store contains gt>f and has{g), then /’s presence follows). Third, a procedural way is to 
dynamically install / at run time; this key aspect originating from FLan enables staged configuration as 
known from dynamic software product lines 113 211. Building on FLan, PFLan combines these three 
ways in an elegant and consistent manner. The introduction of action rates in PFLan moreover allows 
one to specify probabilistic aspects of SPL models such as the behaviour of the user of a product and the 
likelihood of installing a certain feature at a specific momenf wifh respecf fo fhaf of ofher feafures. 


4 A PFLan Model of the Example Product Line 

Fig. shows a specification of fhe family of coffee machines in PFLAN. Fragmenf F is composed of 
sfore S and a process Q. The latter specifies an inifial configurafion phase D, during which all primifive 
feafures excepf ringtone can be insfalled (fhe order of insfallafion is influenced by fhe relative weigh! of 
fhe feafures, more on fhis below). This phase ends af a cerfain momenf when a specific producf (coffee 
machine) is said fo be pre-configured, modeled by fhe insfallafion of an ad-hoc defined feafure pre-conf, 
thus initiating the execution of process R, which specifies the product’s run-time behaviour. Note that it 
is specifically allowed to install (or bind) a feature at run time (cf. ringtone in our toy example). 

The store, instead, is made up of two parts: constraints derived from the requirements (^i) plus 
contextual information ( 82 )- The current action constraints are quite simple (all are of the form do(f) —)• 
has{g)) but, as said before, they could be more sophisticated upon need (e.g. the constraint on action 
cappuccino could be specified as do{cappuccino) —> has{cappuccino) A has{ringtone) to require not 
only the presence of its corresponding feature but also that of the ringtone feature). In Fig. a product 
line of European coffee machines is instantiated by the explicit context information in{Europe). 

The configuration process D is a simple rated choice among the installation of some of the features 
a coffee machine may exhibit. This specifies a sort of race between features and may be thought of as 
independent designers competing to install the features for which they are responsible. The semantics 
of PFLan ensures that all executions will result in a consistent configuration if the process begins with 
a consistent store, i.e. the semantics forbids the installation of features that are mutually exclusive or 
prohibited by (a combination of) the constraints. Formally, multiple installations of the same feature 
does not have any effect, as installed features are organised in a set. The rates of the actions influence 
fhis race by defermining a higher (or lower) probabilify for fhe insfallafion of one feafure wifh respecf 
fo anofher (or prior fo anofher). In our example, fo reflecf fhe facf fhaf coin and sugar are core feafures, 
we assign higher rafes fo fhem fhan fo fhe opfional feafures fo raise fheir chances of being insfalled 
firsf. Moreover, since we are modelling a coffee machine and since coffee is a necessary ingredienf 
for cappuccino, we assign a higher rale fo fhe feafure coffee fhan fo Ihose of ofher drinks. As a resull, 
fhe probabilify fo inslall sugar in fhe firsf step, given fhaf also pre-conf, euro, cancel, coffee, tea and 
cappuccino can be insfalled, fhus becomes 10 - 1 - 10 - 1 - 10 + 7 + 9 - 1 - 6+3 = V"- 

Process R, finally, describes fhe run-fime execution of a coffee machine. The machine may eifher 
accepl a euro or a dollar, depending on fhe markef if is mean! for. After fhaf, fhe user may cancel 
coin insertion, upon which fhe machine relurns fo ils initial slafe and (usually) refurns fhe coin. Wifh 
a probabilify of 1/11, however, fhe machine does nol relurn fhe coin (viz. jq ^). If coin insertion is nol 
canceled, fhe user may (P 2 ) or may nol (P 3 ) push a button for sugar. In case sugar is selected, if is also 
poured, afler which fhe user can selecf a beverage. Buf, wifh a probabilify of = '/e fhe machine 
is oul of sugar, after which fhe user may eifher cancel fhe coin insertion or go for an unsugared drink. 
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F = [5ie] 

5 = 5i52 

= has (euro) y has (dollar) has (euro) ^ in(Europe) has(dollar) ^ in(Canada) 
has(cojfee) V has(tea) V has(cappuccino) has(tea) —>■ in(Europe) 
dollar 0 euro cappuccino > coffee 

do(euro) —)■ has(euro) do(dollar) —)■ has(dollar) 

do(sugar) —> has(sugar) do(ringtone) —>■ has(ringtone) do(cancel) —>■ has(cancel) 

do(pour_sugar) —> has(sugar) do(out_of_sugar) —> has(sugar) 

do(return_coin) —> has(cancel) do(no_return) —> has(cancel) 

do(coffee) —> has(cojfee) do(tea) —> has(tea) do(cappuccino) —> has (cappuccino) 

do(pour_cojfee) —)■ has(cojfee) do(out_of_coffee) —> has(coffee) 

do(pour_tea) —> has(tea) do(out_of_tea) —> has(tea) 

do(pour_milk) —> has (cappuccino) do(out_of_milk) —> has (cappuccino) 

Si = in(Europe) 

Q = D+(install(/>re-con/), 10)./? 

D = (i nsta 11 (ewro), 10).2+ (install((/o/Zar), 10).0+ (i nsta 11 (sMgar), 10).0+ (install(cance/),7).2 
+ (insta 11 (coj^ee),9).2+ (install (tea), 6).2 + (install (ca;5;?Mcdno),3).2 
/? = ((euro,25).Q) + (dollar,25).®)',Pi 

Pq = (return_coin,lO).R +(no_return,l).R 
Pi = (cancel, 5 ).Po+P 2 +P 3 

Pi = (sugar,15).tb', ((pour_sugar,\ 0 ).P 3 -\-(out_of_sugar,2).Pi) 

P 3 = (cojfee,20).Pii-\-(tea,12).P 3 +(cappuccino,2,).P(, 

P 4 = (pour_cojfee,10).P%-\-(out_of_cojfee,2).P 3 
P$ = (pour_tea, 10) .P% + (out_of_tea,2) .P 3 

P(i = (pour_milk, 10).0; ((pour_coffee, 10).P8 + (out_of_cojfee,2).R) + (out_of_milk, 2 ).P 3 
Pg = Pg + (insta\\(ringtone),^).(ringtone, l^).Pg 
P<) = (take_drink,lO).R+(no_cup,l).R 


Figure 5: PFL AN specification of the family of coffee machines (instantiated for Europe) 


Beverage selection (more likely coffee than tea or cappuccino) is followed by the drink being poured 
(again with a probability that the chosen drink is unavailable), which in case of cappuccino concerns 
both milk and coffee. In case coffee or tea was chosen but unavailable, the user can again choose a 
beverage (and the machine may have been refilled). In the specific case that milk was poured but coffee 
is not available, the user has bad luck as the machine returns to its idle state before completing the chosen 
beverage. In case a drink was poured successfully, a ringtone may follow (in which case it first needs to 
be installed). The user then either takes the drink or, with a '/ii probability, realizes that sadly enough 
there was no cup available. Either way, the machine returns to its initial state. 

Note how the rates ‘influence’ the behavior, in the sense that the choice operator is no longer purely 
non-deterministic, but probabilistic, i.e. the rates provide a probabilistic model of the behavior of the 
coffee machine and its environment (the users). Consider, e.g., the choice of a beverage: (coffee, 20) .P 4 + 
(tea, 12).P 5 + (cappuccino, ?i).P(,. The probability to choose coffee is Vi (viz. 204 -^ 2 + 8 ^’ compared to 0.3 
for tea and ‘/s for cappuccino. Similarly, the probability to cancel coin insertion is 5 ^ 15 ^ 20 + 12+8 ~ 

(i.e. rather low). Note that we need to expand processes Pi and P 3 to calculate this probability. 
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The rates that we assigned in this example merely serve to illustrate the proof-of-concept that we 
present in this paper. In practice, those rates may be obtained from a statistical analysis of the actual 
product configuration processes and product behaviours, possibly contained in historical logs. 

Note that D and R are not purely distinct (pre-)configuration and run-time processes, respectively: 
feature ringtone may be installed dynamically at run time (i.e. possibly by R but never by D) and it can 
be thought of as, e.g., a software module. This is an example of a staged configuration process, in which 
some optional features are bound at run time rather than at (pre-)configuration time. 


5 Quantitative Analysis with MultiVeStA 


MultiVeStA p6| is a statistical analysis tool developed and maintained by S. Sebastio and A. Vandin. It 
extends the (distributed) statistical model-checking tools PVeStA Q and VeStA pTj , developed at the 
Department of Computer Science of the University of Illinois at Urbana-Champaign. Differently from 
its predecessors, MultiVeStA can easily be integrated with any formalism which allows for probabilistic 
simulations. It has so far been used to analyse transportation systems 1251, volunteer clouds crowd¬ 
steering pl| and swarm robotic p0| scenarios. 

In this paper, we use MultiVeStA to analyse PFLan specifications in order to obtain statistical es¬ 
timations of quantitative properties expressed in MuItiVeStA’s query language MuItiQuaTEx (an exten¬ 
sion of QuaTEx Q). MultiVeStA provides such estimations by means of distributed statistical analysis 
techniques known from statistical model checking | [28|[2^ . A prototypical tool integrating MultiVe¬ 
StA and PEEan is available at https://code.google.eom/p/multivesta/wiki/PFLan 
together with all files necessary to reproduce the experiments discussed in this section. 

Probabilistic simulations of a PEEan specification can easily be obtained by executing the model 
step-by-step by applying the rules of Eig.[^ each time selecting one of the computed one-step next-states 
according to the probability distribution obtained after normalising the rates of the generated transitions. 
Classical statistical model checking techniques allow one to perform analyses like “is the probability 
that a property holds greater than 0.3?” or “what is the probability that a property is satisfied?” over a 
given specification. Next to performing such kinds of analyses over products, MultiVeStA also allows to 
estimate the expected values of properties that can take on any value from M, like “what is the average 
cost of products generated from a software product line specification?”. Estimations are computed as the 
mean value of n samples obtained from n simulations, with n large enough to grant that the size of the 
(1 — a) X 100% confidence interval (Cl) is bounded by 5. In other words, if a MuItiQuaTEx expression 
is estimated as x, then with probability (1 — a) its actual expected value belongs to the interval \x — ^/ 2 , 
x + ^/i], A Cl is thus specified in terms of two parameters: a and 5. In all experiments discussed in this 
section, we fixed a = 0.1, and 5 = 0.1 and 5 = 0.5 for probabilities and costs of products, respectively. 

MuItiVeStA’s property specification language MuItiQuaTEx is very flexible, based on the following 
ingredients: real-valued observations on the current ‘state’ (e.g. the total cost of installed features), arith¬ 
metic expressions and comparison operators, if-then-else statements, a one-step next operator (which 
triggers the execution of one step of a simulation) and recursion. Intuitively, we can use MuItiQuaTEx to 
associate a value from M to each simulation and subsequently use MultiVeStA to estimate the expected 
value of such number (in case this number is 0 or 1 upon the occurrence of a certain event, we thus 
estimate the probability of such an event to happen). 


6 Quantitative Analyses of the Example Product Line 

Some properties that we can verify over our toy example are as follows: 
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P[ The probability to run into a deadlock before completing the pre-configuration phase; 

P 2 For each of the 8 primitive features (sugar, ringtone, cancel, cappuccino, coffee, tea, dollar, euro), 
the probability to have it installed after the pre-configuration phase or at a given simulation step v; 

P 3 The average cost of products obtained from the pre-configuration phase, or of the ‘intermediate’ 
ones obtained at a given simulation step v. 

Note that we consider any configuration obtained by intermediate steps to be a (possibly intermediate) 
product. This may thus refer to an unfinished product or to underspecified software, or concern a not yet 
fully developed product. When no more features can be installed, we speak of a final product. 

While not explicitly stated, all experiments discussed in this section refer to versions (defined below) 
of the PFLan specification of Fig. [^without the contextual information S 2 = in{Europe), so as to study 
properties of our example without restrictions to a specific context (thus implicitly allowing deadlocks). 

Property Pi is useful for studying the correctness of the PFLan specification of a product line, in this 
case by verifying the probability to successfully complete the pre-configuration phase of a product from 
the product line. Property P 2 is useful for studying how often (on average) a feature is actually installed 
in a product from the product line, which is important information for those designers or programmers 
responsible for the production or programming of a specific feature or software module. Property P 3 , 
finally, is useful for studying the average cost of assembling a product from the product line, based on 
the costs of the features constituting a product defined by the attributed feature model depicted in Fig. [T] 
Listing [T] depicts a MultiQuaTEx expression to evaluate P\. Lines [T]|^ define a recursive temporal 
operator which is evaluated against a simulation: it gives 0.0 if the feature pre-conf is installed in the 
current simulation state (Line|^; it gives 1.0 if the current state is a deadlock (Line[^; or it is recursively 
evaluated in the next simulation state (Line[^. Intuitively, # is the one-step temporal operator, while 
real-valued observations on the current state are evaluate resorting to the keyword s . rval. A number 
of predefined observations is currently supported, e.g. we can query whether a given feature is currently 
installed (as in Line for pre-conf) or whether the current process has no more actions that are allowed 
by the constraints, in which case we say that it is in a deadlock state (Line[^. Finally, Line specifies 
the property to be studied: the expected value of the defined recursive temporal operator. 


1 

2 

3 

4 

5 


Listing 1: The MultiQuaTEx expression corresponding to property P\ 


DeadlockInPreconf() = 
if {s.rval( "pre-conf ") == 1.0} then 0.0 
else if {s.rval{ "deadlock" ) == 1.0} then 1.0 
else #DeadlockInPreconf() fi fi ; 
eval E[ DeadlockBeforePreconf() ] ; 


We evaluated Pi against our PEEan model, obtaining probability 0.0, i.e. the pre-configuration phase 
(almost surely) always terminates. 

Now consider our model to be modified according to Eig. i.e. by replacing F with F', and both Q 
and D with D'. This version still contains a pre-configuration phase {D') followed by the same run-time 
phase {R) of the original model. Essentially, D' tries to install all features, possibly in different orders. 


F' = [5 1 D'\ (install(pre-con/), 10)./?] 

Ff = (install(eMro), 1O).0 || (install((/o/Zar), 1O).0 || (install(™gar), 1O).0 || (install(canceZ),7).0 
II (install(coj^ee),9).0 || (install(tea),6).0 || (install(cappMccZno),3).0 


Eigure 6 : A modified version of the PEEan specification of Eig. [^ 
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By evaluating Pi against the modified version of our model we obtain probability 1.0, i.e. the pre¬ 
configuration phase (almost surely) never terminates. In fact, we can install only one among dollar or 
euro (cf. the first constraint of in Fig.|^, and consequently one of the two installations will never suc¬ 
ceed. Pi can thus indeed be used to check liveness properties of PFLan specifications, e.g. to individuate 
specifications leading, with a certain probability, to deadlocks. 

Listing 1^ depicts a MultiQuaTEx expression to evaluate P 2 and P 3 when considering the products 
obtained after the pre-configuration phase. Such an expression shows how MultiQuaTEx allows one 
to express more properties at once, which can be estimated by MultiVeStA reusing the same simula¬ 
tions. Eines[T]j^define the recursive temporal operator ProductCostAfterPreconf. It is evaluated 
against a simulation as the cost of the product obtained from the pre-configuration phase. As shown in 
Eine a further predefined observation is supported, viz. cost, which provides the cost of the current 
product. Eines [4|6] define a parametric recursive temporal operator which evaluates to 1.0 if the fea¬ 
ture provided as parameter is installed during the pre-configuration phase, and to 0.0 otherwise. Einally, 
Eines [7pT] specify the properties to be analysed: the average cost of products generated by the pre¬ 
configuration phase (Eine and for each of the 8 primitive features the probability to have it installed 
(Eines [8pT] ). We remark that MultiVeStA adopts a procedure which takes into account that each property 
might require a different number of simulations to satisfy the required confidence interval Cl. 


ProductCostAfterPreconf() = 
if {s.rval("pre-conf") == 1.0} then s.rval("cost") 

else #ProductCostAfterPreconf() fi ; 

IsInstalledAfterPreconf(feature) = 
if {s.rval("pre-conf") == 1.0} then s.rval(feature) 

else #l sInstalledAfterPreconf({feature}) fi ; 
eval E[ ProductCostAfterPreconf() ]; 

eval E[ IsInstalledAfterPreconf("sugar") ]; eval E[ IsInstalledAfterPreconf("ringtone") ]; 

eval E[ IsInstalledAfterPreconf("cancel") ]; eval E[ IsInstalledAfterPreconf("cappuccino") ]; 

eval E[ IsInstalledAfterPreconf("coffee") ]; eval E[ IsInstalledAfterPreconf ( "tea" ) ]; 

eval E[ IsInstalledAfterPreconf("dollar") ]; eval E[ IsInstalledAfterPreconf("euro") ]; 


Eisting 2: The MultiQuaTEx expression corresponding to properties P 2 and P 3 

We evaluated the MultiQuaTEx expression of Eistingj^against the original model of Eig. The obtained 
average cost is 14.07, while the probabilities of installing the primitive features are given in the first row 
of Table [T] Clearly, the probability with which features are installed (as well as the average cost of the 
obtained products) is highly affected by the rate at which pre-conf is, installed (10 in Eig. [^: a lower 
or higher rate rate leads to more or less iterations of D, respectively. In order to quantify the influence 
of this rate, we further evaluated the expression of Eisting against the model obtained changing the 
aforementioned rate to 50. The obtained average cost of products is 4.46, while the probabilities of 
installing the features are provided in the second row of Table [T] As expected, the higher installation rate 
of pre-conf has the effect of decreasing the average number of iterations of the pre-configuration phase, 
leading to a lower probability of installation of the features and to a lower average cost of products. 






Eeatures 





Rate of install {pre-conf) 

sugar 

ringtone 

cancel 

cappuccino 

coffee 

tea 

dollar 

euro 

10 

0.49 

0.0 

0.45 

0.13 

0.50 

0.40 

0.33 

0.38 

50 

0.17 

0.0 

0.11 

0.0 

0.14 

0.10 

0.12 

0.13 


Table 1: Probability of installing a feature during the pre-configuration phase of model in Eig.|^(P 2 )- 
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We conclude this section by showing how MultiVeStA can be used to analyse properties of a PFLan 
specification upon varying the number of performed simulation steps. Listing [^sketches how the Multi- 
QuaTEx expression of Listing]^ can be made parametric with respect to a given set of simulation steps. 
First of all, the temporal operators were modified so that they are evaluated with respect to a specific 
step given as parameter. We actually provide only the updated temporal operator regarding the costs of 
products (Lines [T]0, as the other has been modified similarly. Subsequently, it is necessary to specify a 
range of values for the parameter. Lines [ 6 ]|^ specify that we are interested in studying the properties for 
steps going from 0.0 to 40, with an increment of 1.0. 


1 

ProductCostAtStep(step) = 

2 

if (s.rval ("steps") == step) then s.rval ("cost") 

3 

else #ProductCostAfterPreconf(step) 

5 

IsInstalledAtStep(step,feature) = 

6 

eval parametric (E [ ProductCostAtStep (step) ],E[ IsInstalledAtStep (step, "sugar" ) 

7 

E[ IsInstalledAtStep(step, "euro") ] ,step, 0 . 0 , 1 . 0 , 40 . 0 ) ; 


Listing 3: The MultiQuaTEx expression corresponding to P 2 and P 3 upon varying the simulation steps 


We evaluated also the parametric property of Listing against the original model of Fig. All such 
analyses (41 x 9 different properties) were evaluated using the same simulations. The results are pre¬ 
sented in two plots: one for costs (ProductCostAf terPreconf) in Fig.[^ and one for probabilities 
(IsInstalledAfterPreconf) in Fig.[^ 



Figure 7: Average product cost during the first 40 steps 

As expected. Fig. shows that the average cost (on the y-axis) of the intermediate products generated 
from the software product line grows with respect to the number of performed simulation steps. In 
particular, it shows a fast growth during the first 6 steps, reaching an average cost of 13, and then it 
essentially stabilises, eventually reaching its maximum (14.38) from step 26 onwards. This is consistent 
with our PFLan specification, consisting of a pre-configuration phase during which the majority of the 
features are installed, followed by a run-time phase modelling the behaviour of the generated product 
(and possibly installing ringtone). 
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Fig. shows that the probabilities (on the y-axis) for each of the features to be installed evolve 
similarly to the average cost of the generated products, although, clearly, with different scales: they 
show a fast growth during the first 6 steps, after which they essentially stabilise while approaching their 
maximum. The maximum probabilities of installing the various primitive features are as follows: 0.0 for 
ringtone, 0.07 for cappuccino, 0.32 for euro, 0.35 for dollar, 0.38 for tea, 0.43 for cancel, 0.49 for coffee 
and 0.51 for sugar. Note that the probability of installing ringtone is really very low (0.0 or, to be precise, 
its actual expected value belongs to the interval [0,0 +^ 2 ] = [0,0 + = [0,0.05]). Indeed, while the 

installation of ringtone is allowed by our specification, it is optional (except when serving cappuccino). 
Note, however, that we considered simulations consisting of only 40 steps (the x-axes in the two fig¬ 
ures). For longer simulations, we would of course have obtained a higher probability to install ringtone. 



Figure 8: Average feature installation probability during the first 40 steps 


7 Conclusion 


In this paper, we have continued a line of research presented at earlier editions of FMSPLE ||^|^ by 
enriching FLan, a high-level feature-based modelling language for software product lines, with quan¬ 
titative information. The result, PFLan, allows one to model and analyse the likelihood of installing 
features, the probabilistic behaviour of users of products of the product line, and the costs of products, 
next to probabilistic quantifications of ordinary temporal properties (e.g. “what is the probability that cof¬ 
fee is poured while no cup was available?”). In addition, we extended the qualitative analysis framework 
for software product lines implemented in Maude with statistical techniques for quantitative analysis. 

The modelling and analysis capabilities of PFLan were illustrated on a simple product line of coffee 
machines. In the future, we plan to investigate the scalability of our (tool) framework by consider¬ 
ing more realistic and complex scenarios. We also intend to add the possibility to define quantitative 
constraints to PFLan, possibly by adopting further operations from extensions of the concurrent con¬ 
straint paradigm that can deal with quality of service and mobility 0 and its stochastic extension p^. 
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Both the check operation of concurrent constraint programming, to prevent inconsistencies, and its re¬ 
tract operation, to remove (syntactically present) constraints from the store, might be useful to enable 
the dynamic (un)installation of features in the presence of (soft) quantitative constraints (i.e. not only 
Boolean p0|). In particular, we would like to investigate the consequences of allowing the explicit unin¬ 
stallation of a feature, e.g. due to its malfunctioning or due to the need of replacing it by a better (version 
of the) feature. Such features were shown successful in services computing for the specification of 
service-level agreements and negotiation processes p3| . Finally, we would like to allow behaviour that 
is explicitly influenced by the constraint store, as in 1121. In our example, this would allow us to model, 
e.g., the probability of a user choosing a coffee to depend on the location of the coffee machine (i.e. 
Europe or Canada), thus allowing us to assign, e.g., a higher weight to ordering cappuccino in Europe. 
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